Redesigning Programs and Unit Assessments

Chris Rust, Emeritus Professor of Higher Education, Oxford Brookes University, UK, shared his thoughts and insights into improving programs and unit assessments at the AGSM seminar Redesigning course assessment; I’ve summarised the key concepts.

THE PROBLEM

There are problems with current assessment practice. Essentially they are a lack of alignment and coherence between unit assessments and program learning outcomes. Lipman in Thinking in Education (2013) summarises the problem as “Students acquire bits of knowledge that, like ice cubes frozen in their trays, remain inert & incapable of interacting with one another” (p54).

THE SOLUTION

Program focussed assessment should be “specifically designed to address major programme outcomes rather than very specific or isolated components of the course. It follows then that such assessment is integrative in nature, trying to bring together understanding and skills in ways which represent key programme aims [valid]. As a result, the assessment is likely to be more authentic and meaningful [relevant] to students, staff and external stakeholders” PASS Project Position Paper

The PASS Paper provides a useful mind map of the impact of program assessment.

Another solution is to re-evaluate summative and formative assessments in units. To move towards less but better summative assessments, and more formative assessments to develop students’ assessment literacy.

HOW TO DO BETTER PROGRAM FOCUSED ASSESSMENT

We are familiar with constructive alignment of learning outcomes, learning activity and assessment as outlined by Biggs. This concept is applied to a program by O’Neill, Donnelly & Fitzmaurice (2014) through “curriculum sequencing”. This occurs through the:

1. Development of a collective philosophy that underlies the program. For example, “the program’s philosophy is XYX and the units and sequencing reflective this through ABC…”
2. Communication of this philosophy and sequence to both staff and students. Students come to the program because they chose this philosophy, it is seen as the motivation or rationale to select this program amongst many. Also, so that not only existing staff but future staff that teach into the program are aware of, and support the philosophy.
3. Development of strong “building blocks” in the curriculum through capstones and cornerstone (capstones throughout the curriculum). This helps to consolidate the learning at various stages and prevents the atomisation of learning and the “ice cube accumulation of knowledge”.

Chris referred to Alverno College in the US as a good example of this approach.

HOW TO DO BETTER SUMMATIVE ASSESSMENT

Chris contends that awarding students percentages for their performance in an assessment is essentially unreliable and inconsistent. He quoted studies providing evidence of not only inconsistency between markers marking the same paper but inconsistency when a marker marks the same paper twice (Hartog & Rhodes, 1935). He argues for a pass/fail approach to assessment, being more reliable and moving away from the currency of marks. He says “We’ve created a phoney currency called marks where students won’t do anything unless we ‘pay’ them with marks.”  

One of the central ideas in Chris’ presentation is that to improve learning, students must be brought into the community of assessment practice. This can be illustrated in the diagram below.

The aim of which is so “that the student comes to hold a concept of quality roughly similar to that help by the teacher” (Sadler, 1989). Chris contends that the student at the end of their degree must be able to evaluate work and decide if it is to the standard required of that discipline. To achieve this there need to be higher levels of student involvement in their learning by student/staff and student/student interactions. Feedback must not be passively given, it needs to be provided as an ongoing dialogue, not only between the teacher and student but between the student and their peers. This then introduces the concept of assessment literacy of students.

WHAT IS ASSESSMENT LITERACY?

To be assessment literate students have:

1. An appreciation of assessments’ relationship to learning.
2. An understanding of the nature, meaning and level of assessment criteria and standards.
3. Skills in self and peer assessment.

ACTIVE ENGAGEMENT WITH CRITERIA

The aim here is to get students actively using the criteria through a marking exercise, self-assessment, peer feedback and assessment.

Chris provides an example of how to do this below.

ACTIVE ENGAGEMENT WITH FEEDBACK (IMPROVE FEEDBACK-PREPARE STUDENTS)

To improve feedback students must be prepared. To do this:

1. Align expectations (of staff, students and the team of markers) about the purpose of feedback. What will feedback be provided on? How will it contribute to student learning? How will the student apply it?
2. Students often don’t realise that they are being provided with feedback, especially oral. A suggestion is to identify all feedback provided by the words “Here is some feedback…”
3. Model the application of feedback. For example using previously marked assignments to show how feedback was used.
4. Require and develop student self-assessment.
5. Perhaps the most interesting one is to encourage the application of feedback. For example, in a subsequent piece of work, the student is required to show how they have used prior feedback to improve their work. You could develop a simple template for the student. At the bare minimum students will actually have to go back and look at the feedback to complete this template!

My previous feedback	How I applied that feedback to this task

ACTIVE ENGAGEMENT WITH FEEDBACK (IMPROVE FEEDBACK-ENSURE IT’S FIT FOR PURPOSE)

To improve feedback, ensure it’s fit for purpose. To do this:

1. Ensure students have MOM (Motive, Opportunity, Means) to engage with feedback. Have you designed your assessments in a manner that provides students with the motive to engage with feedback? Do they have an opportunity in your unit to apply this feedback? Do they have the means (skills or knowledge) to apply and engage with this feedback? Have you provided instructions and tasks where they can develop these skills?
2. Consider an assessment pathway of draft-plus-rework, where the feedback effort (for students and markers) is at the draft stage and summative grade given for final work.
3. Consider adaptive release of grades. Students need to engage with feedback, whether it’s at a superficial level of merely viewing it or more in depth where they have to reflect on it before they can access their grades.

IN SUMMARY

Formative assessment in a unit should be designed so that:

• Feedback is an effect, not an input into the unit.
• Feedback should have an impact on students, furthering their learning of a skill or knowledge.
• There are events and feedback loops. Where in the unit are students particularly challenged (‘troublesome knowledge’, epistemological jumps)? What loops can you design, such as students self-assessing, or engaging with and using the feedback to work on improved drafts etc?
• Self and peer assessment need to be seen as graduate capabilities.
• Feedback needs to be an ongoing dialogue between the student, their peers and the teacher.
• Bring the student into the community of assessment practice.

To access the original slides of Chris’ presentation follow this link.